W7. Eigenvalue Diagonalization
1. Theory
1.1 Introduction to Eigenvalues and Eigenvectors
In linear algebra, eigenvalues and eigenvectors represent one of the most powerful concepts for understanding the structure and behavior of matrices. At their core, eigenvalues and eigenvectors answer a fundamental question: Are there any special directions in space where a linear transformation behaves like simple scaling?
When we apply a matrix
Definition (Eigenvector and Eigenvalue): Let
The scalar
This equation is the defining property: when we multiply
Geometric Intuition: Imagine a matrix as a transformation that stretches, rotates, and shears space. An eigenvector points in a direction that is special for this transformation: applying the transformation along this direction is equivalent to scaling. This makes eigenvectors the “natural axes” or “principal directions” of the transformation.
1.1.1 Simple Examples
Let’s verify with a concrete example. Consider the matrix
Compute
Since
1.2 Finding Eigenvalues: The Characteristic Equation
To find eigenvalues systematically, we use the characteristic equation. Starting from the definition
where
This equation is called the characteristic equation of
Algorithm to Find Eigenvalues:
- Compute the characteristic polynomial: Calculate
. - Set equal to zero: Solve
. - Find the roots: The roots of the characteristic polynomial are the eigenvalues of
.
Important Note: The Fundamental Theorem of Algebra guarantees that over the complex numbers
Special Case—Triangular Matrices: If
Therefore, the eigenvalues of a triangular matrix are exactly its diagonal entries.
1.3 Finding Eigenvectors
Once we have found an eigenvalue
The set of all solutions to this equation forms the eigenspace for eigenvalue
Algorithm to Find Eigenvectors:
- Form the matrix
by subtracting from the diagonal entries of . - Row reduce
to find its null space. - Express solutions parametrically in terms of free variables.
- Identify basis vectors for the null space—these are the eigenvectors.
1.4 Trace and Determinant Relationships
Two important relationships connect eigenvalues to properties we can compute directly from the matrix:
Trace Property: The sum of all eigenvalues (counted with algebraic multiplicity) equals the trace of
Determinant Property: The product of all eigenvalues (counted with algebraic multiplicity) equals the determinant of
These properties provide useful checks when computing eigenvalues by hand. If your computed eigenvalues don’t satisfy these relationships, you’ve made an error.
1.5 Linear Independence of Eigenvectors
A crucial property that guarantees nice structure is:
Theorem: If
This means that eigenvectors corresponding to different eigenvalues can never be linearly dependent. This property is fundamental to diagonalization.
1.6 Introduction to Diagonalization
The ultimate goal of eigenvalue analysis is often diagonalization—rewriting the matrix in a simpler form using its eigenvalues and eigenvectors.
An
Equivalently,
What are
When Can We Diagonalize? The key requirement is that we need
- Case 1:
has distinct eigenvalues. In this case, the corresponding eigenvectors are automatically linearly independent. - Case 2:
has repeated eigenvalues. We must check that for each repeated eigenvalue , the number of linearly independent eigenvectors equals its multiplicity in the characteristic polynomial. This is checked by examining the dimension of the eigenspace.
A matrix that has repeated eigenvalues but fewer than
1.6.1 Why Diagonalization is Useful
Diagonalization is powerful because it simplifies matrix computations:
- Powers of matrices: If
, then . Since is diagonal, is easy to compute: . - Solving differential equations: Systems of differential equations
are easily solved when is diagonalized. - Understanding transformations: Diagonalization reveals the “principal axes” along which the transformation acts simply as scaling.
1.7 Spectral Properties
Several fundamental theorems describe special properties related to eigenvalues:
Theorem (Cayley-Hamilton): Every square matrix satisfies its own characteristic equation. If
Theorem (Spectral Mapping): If
Theorem (Symmetric Matrices): If
- All eigenvalues are real numbers
- Eigenvectors corresponding to distinct eigenvalues are orthogonal
is always diagonalizable by an orthogonal matrix :
1.8 Special Cases and Applications
Understanding eigenvalues is essential across mathematics and science:
- Physics: Eigenvalues represent natural frequencies of vibrating systems, principal stresses in materials, and energy levels in quantum mechanics.
- Computer Science: PageRank (Google’s algorithm) is based on eigenvalues, as is facial recognition and image compression.
- Data Science: Principal Component Analysis (PCA), a fundamental technique in machine learning, uses eigenvalues to reduce dimensionality.
- Economics: Eigenvalue analysis models input-output systems and portfolio optimization.
2. Definitions
- Eigenvalue: A scalar
such that for some nonzero vector of matrix . - Eigenvector: A nonzero vector
such that for some scalar (eigenvalue) of matrix . - Characteristic Polynomial: The polynomial
for an matrix . - Characteristic Equation: The equation
, whose roots are the eigenvalues of . - Eigenspace: The set of all eigenvectors (plus the zero vector) corresponding to a given eigenvalue
; equivalently, the null space of . - Algebraic Multiplicity: The multiplicity of an eigenvalue
as a root of the characteristic polynomial. - Geometric Multiplicity: The dimension of the eigenspace of eigenvalue
(i.e., the number of linearly independent eigenvectors for that eigenvalue). - Diagonalizable Matrix: A square matrix
for which there exists an invertible matrix and diagonal matrix such that . - Defective Matrix: A square matrix that has repeated eigenvalues but is not diagonalizable (geometric multiplicity < algebraic multiplicity for some eigenvalue).
- Trace of a Matrix: The sum of the diagonal entries:
. This equals the sum of eigenvalues. - Similar Matrices: Two matrices
and are similar if there exists an invertible matrix such that . Similar matrices have the same eigenvalues. - Orthogonal Matrix: A square matrix
such that , i.e., . - Spectral Theorem: The theorem stating that every real symmetric matrix is diagonalizable by an orthogonal matrix.
3. Formulas
- Eigenvalue-Eigenvector Relationship:
(defining equation) - Characteristic Equation:
- Finding Eigenvectors:
(solve this homogeneous system) - Trace and Sum of Eigenvalues:
- Determinant and Product of Eigenvalues:
- Diagonalization:
where and - Powers via Diagonalization:
where - Inverse via Diagonalization:
where (if is invertible) - Spectral Decomposition (Symmetric Matrices):
where is orthogonal and is diagonal - Eigenvalue of Matrix Power: If
is an eigenvalue of , then is an eigenvalue of - Eigenvalue of Inverse: If
is an eigenvalue of an invertible matrix , then is an eigenvalue of - Cayley-Hamilton Theorem:
where is the characteristic polynomial
4. Practice
4.1. Eigenvalue Properties (Lab 6, Task 1)
Find the eigenvalues of the matrix
Click to see the solution
Key Concept: The characteristic equation
- Set up the characteristic equation:
- Expand the determinant:
- Solve:
- Verify trace property:
- Verify determinant property:
Answer:
4.2. Eigenvector Verification (Lab 6, Task 2)
Part A. For matrix
Part B. For matrix
Click to see the solution
Key Concept: A vector
Part A:
- Compute
: - Conclusion: Yes,
is an eigenvector with .
Part B:
- Compute
: - Conclusion: Yes,
is an eigenvector with .
Answer: (A)
4.3. Construct Matrix from Eigenvalues (Lab 6, Task 3)
Give three examples of
Click to see the solution
Key Concept: The eigenvalues of a triangular (or diagonal) matrix are its diagonal entries. Any matrix similar to a diagonal matrix with entries 4 and 5 also has those eigenvalues, giving infinitely many solutions.
- Diagonal matrix:
- Upper triangular matrix:
- Lower triangular matrix:
Answer: Three valid examples are
4.4. 3×3 Eigenvalues and Eigenvectors (Lab 6, Task 4)
Find the eigenvalues and eigenvectors of:
Click to see the solution
Key Concept: For a 3×3 matrix, compute the characteristic polynomial by cofactor expansion, factor to find eigenvalues, then solve
Step 1: Compute the characteristic polynomial
Expanding along the first column:
Step 2: Factor and find eigenvalues
Therefore:
Step 3: Eigenvectors for
Row-reduce
Free variable
Step 4: Eigenvectors for
Row-reduce
Free variable
Answer:
4.5. Diagonalization (Lab 6, Task 5)
Diagonalize the matrix
Click to see the solution
Key Concept: To diagonalize
Step 1: Find eigenvalues
Therefore:
Step 2: Eigenvector for
Step 3: Eigenvector for
Step 4: Form
Answer:
4.6. Matrix Power via Diagonalization (Lab 6, Task 6)
For
Click to see the solution
Key Concept: If
Step 1: Find eigenvalues
Therefore:
Step 2: Find eigenvectors
For
For
Step 3: Form the diagonalization
Step 4: Compute
Answer:
4.7. Find Eigenvalues of Triangular Matrix (Assignment 6, Task 1)
Find the eigenvalues and corresponding eigenvectors of the matrix:
Click to see the solution
Key Concept: For a 2×2 matrix, compute the characteristic polynomial, solve for eigenvalues, then find eigenvectors for each eigenvalue.
Step 1: Find eigenvalues
Therefore:
Step 2: Find eigenvector for
Solve
This gives:
Step 3: Find eigenvector for
Solve
This gives:
Answer: Eigenvalues are
4.8. Eigenspace and Algebraic/Geometric Multiplicity (Assignment 6, Task 2)
For the matrix
- Find the characteristic polynomial of
. - Find all eigenvalues of
. - For each eigenvalue, find a basis for the corresponding eigenspace.
Click to see the solution
Key Concept: The characteristic polynomial is found via determinant. Eigenvalues are its roots. For each eigenvalue, the eigenspace is the null space of
Step 1: Find the characteristic polynomial
Since the first column has two zeros, expand along it:
Step 2: Find eigenvalues
From
Step 3: Find eigenspace for
Solve
From the second row:
Basis for eigenspace:
Step 4: Find eigenspace for
Solve
From the first row:
Basis for eigenspace:
Answer: (a) Characteristic polynomial:
- For
: - For
:
4.9. Construct Matrix from Eigenvectors and Eigenvalues (Assignment 6, Task 3)
Let
Click to see the solution
Key Concept: Use the defining equations
Write out the eigen-equations: Let
. Then:Expand first equation:
This gives:
andExpand second equation:
This gives:
andSolve the system: From equations 1 and 3:
- Adding:
- Then:
From equations 2 and 4:
- Adding:
- Then:
- Adding:
Form the matrix:
Answer:
4.10. Verify Eigenvalue-Eigenvector Pair (Assignment 6, Task 4)
For matrix
Click to see the solution
Key Concept: To verify an eigenvalue-eigenvector pair, compute
- Compute
: - Calculate each component:
- First row:
- Second row:
- Third row:
- First row:
- Result:
- Check against
: - Compare:
Answer: The pair
4.11. Prove Eigenvalue Property for Invertible Matrices (Assignment 6, Task 5)
Prove that if
Click to see the solution
Key Concept: Use the definition of eigenvalue and properties of matrix inverses to transform the equation for
Proof:
- Start with the given condition: Suppose
is an eigenvalue of invertible matrix with eigenvector . Then: - Note that
: Since is invertible, . But , so no eigenvalue can be zero. - Multiply both sides by
: - Divide both sides by
(nonzero): - Rearrange to standard form:
- Conclusion: This shows that
is an eigenvector of with eigenvalue .
Answer: If
4.12. True or False: Eigenvalue Properties (Assignment 6, Task 6)
Determine whether each statement is true or false. Justify your answer.
(a) If 0 is an eigenvalue of
(b) If
(c) A matrix can have different eigenvalues with the same eigenvector.
(d) If
Click to see the solution
Key Concept: Reason through each statement using the definitions and properties of eigenvalues and eigenvectors.
(a) TRUE:
If 0 is an eigenvalue of
This means
(b) TRUE:
If
So
(c) FALSE:
Suppose
Then:
Since
(d) FALSE:
Counter-example: Consider
For
However,
Answer: (a) TRUE, (b) TRUE, (c) FALSE, (d) FALSE.
4.13. Determine Diagonalizability (Assignment 6, Task 7)
Describe all matrices
Click to see the solution
Key Concept: A diagonalizing matrix
Step 1: Find eigenvalues
Since
Step 2: Find eigenvector for
Solve
From the second row:
Step 3: Find eigenvector for
Solve
From the first row:
Step 4: Characterize all diagonalizing matrices
Since any nonzero scalar multiple of an eigenvector is still an eigenvector (for the same eigenvalue), the diagonalizing matrices are:
The first column is
Answer: All matrices of the form
4.14. Identify Non-Diagonalizable Matrix (Assignment 6, Task 8)
Which of these matrices cannot be diagonalized?
Click to see the solution
Key Concept: A matrix is diagonalizable if it has
Analysis of
Find eigenvalues:
So
Find eigenvectors: Solve
There is only one linearly independent eigenvector, but we need two. Thus
Analysis of
Eigenvalues (diagonal entries, since upper triangular):
Since eigenvalues are distinct, the corresponding eigenvectors are automatically linearly independent. Thus
Analysis of
Eigenvalues (diagonal entries, since upper triangular):
Find eigenvectors: Solve
From the second row:
There is only one linearly independent eigenvector for the repeated eigenvalue. Thus
Answer:
4.15. Verify Eigenvalue and Find Eigenvalues (Lecture 6, Example 1)
Find the eigenvalues of the matrix
Click to see the solution
Key Concept: The characteristic equation
Set up the characteristic equation:
Calculate the determinant:
Use the quadratic formula:
Therefore:
andVerify trace property:
Verify determinant property:
Answer: Eigenvalues are
4.16. Determine if a Vector is an Eigenvector (Lecture 6, Example 2)
Consider the matrix
Also consider the matrix
Click to see the solution
Key Concept: A vector is an eigenvector if
(a) For matrix
Compute
:Check if this is a scalar multiple of
:Yes! This means
.
(b) For matrix
Compute
:Check if this is a scalar multiple of
:Yes! This means
.
Answer:
- Yes,
is an eigenvector of with eigenvalue . - Yes,
is an eigenvector of with eigenvalue .
4.17. Find Three Matrices with Specified Eigenvalues (Lecture 6, Example 3)
Find three
Click to see the solution
Key Concept: There are infinitely many matrices with the same eigenvalues. We can construct them using triangular or diagonal matrices, where eigenvalues appear on the diagonal.
This problem has infinitely many solutions because the eigenvalues alone don’t determine the matrix uniquely. The easiest examples use triangular or diagonal matrices:
- Diagonal matrix:
Eigenvalues are clearly 4 and 5 (diagonal entries). - Upper triangular matrix:
where can be any nonzero number (e.g., ). Eigenvalues are 4 and 5 (diagonal entries). - Lower triangular matrix:
where can be any nonzero number. Eigenvalues are 4 and 5 (diagonal entries).
Answer: Three possible matrices are:
(Many other answers are possible.)
4.18. Find Eigenvalues and Eigenvectors of a 3×3 Matrix (Lecture 6, Example 4)
Find the eigenvalues and eigenvectors of the matrix:
Click to see the solution
Key Concept: For a 3×3 matrix, we compute the characteristic polynomial using cofactor expansion, find the roots (eigenvalues), then solve the system
Step 1: Calculate the characteristic polynomial
Expand along the first column:
Step 2: Factor and find eigenvalues
Therefore:
Step 3: Find eigenvectors for
Solve
Row reduce to RREF:
In parametric form:
Step 4: Find eigenvectors for
Solve
Row reduce to RREF:
In parametric form:
Answer: Eigenvalues are
4.19. Diagonalize a 2×2 Matrix (Lecture 6, Example 5)
Diagonalize the matrix:
Click to see the solution
Key Concept: Diagonalization requires finding eigenvalues and eigenvectors, then forming matrices
Step 1: Find eigenvalues
Therefore:
Step 2: Find eigenvectors for
Solve
This gives:
Step 3: Find eigenvectors for
Solve
This gives:
Step 4: Form the diagonalization
To find
Answer:
4.20. Use Diagonalization to Compute Matrix Powers (Lecture 6, Example 6)
For the matrix
Click to see the solution
Key Concept: Using diagonalization,
Step 1: Find eigenvalues
Using trace and determinant:
So
Solving
Therefore:
Step 2: Find eigenvectors for
Solve
This gives:
Step 3: Find eigenvectors for
Solve
This gives:
Step 4: Form the diagonalization
Step 5: Compute
Therefore:
(The numerical computation of
Answer:
4.21. Eigenvalues of a Triangular Matrix — Trace and Determinant Check (Tutorial 6, Task 1)
Find the eigenvalues of the matrix:
Click to see the solution
Key Concept: For a triangular matrix, the eigenvalues are the diagonal entries directly.
Read off eigenvalues from the diagonal:
Verify trace:
Verify determinant:
Answer:
4.22. Eigenvalues and Eigenvectors of a Symmetric 3×3 Matrix (Tutorial 6, Task 2)
Find the eigenvalues and eigenvectors of:
Click to see the solution
Key Concept: Compute the characteristic polynomial
Characteristic polynomial:
Expanding along the second row:Eigenvalues:
(algebraic multiplicity 2), (multiplicity 1).Eigenvectors for
: Solve : Two independent eigenvectors: , .Eigenvectors for
: Solve : Eigenvector: .
Answer:
4.23. Characteristic Polynomial Matching (Tutorial 6, Task 3)
Find the values of
Click to see the solution
Key Concept: This is a companion matrix. For such a matrix, the characteristic polynomial is directly
Compute
by cofactor expansion along the first column:Match coefficients with
:
Answer:
4.24. Eigenvectors vs. Eigenvalues — Fill in the Blanks (Tutorial 6, Task 4)
Fill in the blanks:
(a) If you know that
(b) If you know that
Click to see the solution
Key Concept: The eigenvector equation
(a) Compute
(b) Solve the homogeneous linear system
Answer: (a) Compute
4.25. All Eigenvalues and Two Diagonalizing Matrices (Tutorial 6, Task 5)
Find all eigenvalues and eigenvectors of:
Click to see the solution
Key Concept: This rank-1 matrix has eigenvalues 0 (with geometric multiplicity 2) and
Eigenvalues:
, (rank 1). Characteristic polynomial: , so , .Eigenvectors for
: Null space of : Two free variables; two independent eigenvectors, e.g.: , .Eigenvectors for
: gives , so .Two diagonalizing matrices (any choice of two independent eigenvectors for
):
Answer:
4.26. Prove a Matrix Power Formula via Diagonalization (Tutorial 6, Task 6)
Diagonalize
Click to see the solution
Key Concept:
Eigenvalues:
, so , .Eigenvectors:
: :
Matrices:
Compute
:
Answer:
4.27. All Matrices That Diagonalize a Given Matrix (Tutorial 6, Task 7)
Describe all matrices
Click to see the solution
Key Concept: Any invertible matrix whose columns are eigenvectors of
- Eigenvalues of
: Diagonal entries , . - Eigenvectors:
: :
- All diagonalizing matrices for
: Any with (scalar multiples of eigenvectors in each column, keeping invertible). - For
: has the same eigenvectors as (with eigenvalues and ). So the same set of matrices diagonalizes .
Answer: All invertible matrices with columns proportional to
4.28. Which Matrix Cannot Be Diagonalized? (Tutorial 6, Task 8)
Which of the following matrices cannot be diagonalized?
Click to see the solution
Key Concept: A matrix is diagonalizable if and only if for each eigenvalue, the geometric multiplicity equals the algebraic multiplicity.
Answer: